fix(provider): ignore SSE comment heartbeats for chunk timeout - #43607
fix(provider): ignore SSE comment heartbeats for chunk timeout#436071052326311 wants to merge 3 commits into
Conversation
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
|
Current head |
|
Follow-up head |
…ray semantics Deviations from the ported upstream hunks, per review: - anomalyco#43881: the empty-stream guard now fails only when the attempt produced no text/reasoning delta and no tool-call event. Providers may stream real content but omit the usage block and finish reason; retrying those would duplicate already-persisted output (up to 6 attempts). Empty-content deltas still count as empty, so the original clean-EOF retry is intact. - anomalyco#38939: listeners are an Array again. The Set container silently deduplicated identical callback registrations and removed every entry on the first unsubscribe; duplicates must deliver independently. - adds the missing multi-byte UTF-8 split regression for anomalyco#43607's streaming TextDecoder
…co#42150 anomalyco#42176 anomalyco#43881 anomalyco#43607) - O(N) text/reasoning delta accumulation instead of O(N^2) string concat (anomalyco#42150) — the lazy chunk buffer joins on read - finish reason 'error' is set when a stream fails mid-flight (anomalyco#42176) - clean-EOF empty provider streams retry like transient errors (anomalyco#43881), narrowed from the upstream patch: only an attempt that produced no text/reasoning delta and no tool call qualifies, so providers that stream content but omit usage/finish are not retried into duplicate output - SSE comment heartbeats no longer reset the chunk timeout (anomalyco#43607); the streaming TextDecoder handles multi-byte characters split across reads (regression covered)
Issue for this PR
Closes #43519
Type of change
What does this PR do?
chunkTimeoutcurrently restarts for every response body read. SSE comment frames such as: keepalivetherefore prevent an otherwise stalled model stream from timing out. This keeps one deadline across reads and resets it only after a complete SSE event containingdata:. Header timeouts and non-SSE responses are unchanged.The regression uses the existing loopback OpenAI-compatible server: it sends one data event, then comment heartbeats every 20 ms. With a 50 ms timeout, the stream must produce the existing typed response-stream error.
How did you verify your code works?
3a31c4ea801915c0b050df4b3842997ea62b6e938de252f7aed6b8fc4d93a650be397fd385c7c433The upstream
test,typecheck, andnix-evalworkflows require maintainer approval for this fork head.Screenshots / recordings
Not applicable; this is a provider timeout fix.
Checklist